Crowd detection, analysis, and categorization

ABSTRACT

A method, computer system, and a computer program product for analyzing a crowd using a plurality of images captured by an aerial drone is provided. The present invention may include determining a geographic area associated with the crowd. The present invention may also include partitioning the determined geographic area into a plurality of zones. The present invention may then include determining a flight path covering each zone within the plurality of zones. The present invention may further include receiving the plurality of images from the aerial drone. The present invention may also include analyzing the received plurality of images to identify a plurality of individuals associated with the crowd. The present invention may then include predicting a plurality of crowd characteristics based on the analyzed plurality of images. The present invention may further include performing an action in response to the predicted plurality of crowd characteristics.

BACKGROUND

The present invention relates generally to the field of computing, andmore particularly to crowd analysis.

Photography and videography-based applications have gained popularity inthe past decade for administrative and security purposes resulting inthe prevalence of security cameras and equipment in public areas.Existing applications may mostly be based on fixed imagery angles frompre-aligned cameras or cameras with limited movement. Furthermore,camera output may require manual observation to derive information or tomake decisions.

SUMMARY

Embodiments of the present invention disclose a method, computer system,and a computer program product for analyzing a crowd using a pluralityof images captured by an aerial drone. The present invention may includedetermining a geographic area associated with the crowd. The presentinvention may also include partitioning the determined geographic areainto a plurality of zones. The present invention may then includedetermining a flight path covering each zone within the plurality ofzones. The present invention may also include sending the determinedflight path to the aerial drone. The present invention may furtherinclude receiving the plurality of images from the aerial drone. Thepresent invention may also include analyzing the received plurality ofimages to identify a plurality of individuals associated with the crowd.The present invention may then include predicting a plurality of crowdcharacteristics based on the analyzed plurality of images. The presentinvention may further include performing an action in response to thepredicted plurality of crowd characteristics.

BRIEF DESCRIPTION OF THE SEVERAL VIEWS OF THE DRAWINGS

These and other objects, features and advantages of the presentinvention will become apparent from the following detailed descriptionof illustrative embodiments thereof, which is to be read in connectionwith the accompanying drawings. The various features of the drawings arenot to scale as the illustrations are for clarity in facilitating oneskilled in the art in understanding the invention in conjunction withthe detailed description. In the drawings:

FIG. 1 illustrates a networked computer environment according to atleast one embodiment;

FIG. 2 illustrates a networked aerial image capture device systemaccording to at least one embodiment;

FIG. 3 is an operational flowchart illustrating a process for crowdanalysis according to at least one embodiment;

FIG. 4 is a block diagram of internal and external components ofcomputers and servers depicted in FIG. 1 according to at least oneembodiment;

FIG. 5 is a block diagram of an illustrative cloud computing environmentincluding the computer system depicted in FIG. 1, in accordance with anembodiment of the present disclosure; and

FIG. 6 is a block diagram of functional layers of the illustrative cloudcomputing environment of FIG. 5, in accordance with an embodiment of thepresent disclosure.

DETAILED DESCRIPTION

Detailed embodiments of the claimed structures and methods are disclosedherein; however, it can be understood that the disclosed embodiments aremerely illustrative of the claimed structures and methods that may beembodied in various forms. This invention may, however, be embodied inmany different forms and should not be construed as limited to theexemplary embodiments set forth herein. Rather, these exemplaryembodiments are provided so that this disclosure will be thorough andcomplete and will fully convey the scope of this invention to thoseskilled in the art. In the description, details of well-known featuresand techniques may be omitted to avoid unnecessarily obscuring thepresented embodiments.

The present invention may be a system, a method, and/or a computerprogram product at any possible technical detail level of integration.The computer program product may include a computer readable storagemedium (or media) having computer readable program instructions thereonfor causing a processor to carry out aspects of the present invention.

The computer readable storage medium can be a tangible device that canretain and store instructions for use by an instruction executiondevice. The computer readable storage medium may be, for example, but isnot limited to, an electronic storage device, a magnetic storage device,an optical storage device, an electromagnetic storage device, asemiconductor storage device, or any suitable combination of theforegoing. A non-exhaustive list of more specific examples of thecomputer readable storage medium includes the following: a portablecomputer diskette, a hard disk, a random access memory (RAM), aread-only memory (ROM), an erasable programmable read-only memory (EPROMor Flash memory), a static random access memory (SRAM), a portablecompact disc read-only memory (CD-ROM), a digital versatile disk (DVD),a memory stick, a floppy disk, a mechanically encoded device such aspunch-cards or raised structures in a groove having instructionsrecorded thereon, and any suitable combination of the foregoing. Acomputer readable storage medium, as used herein, is not to be construedas being transitory signals per se, such as radio waves or other freelypropagating electromagnetic waves, electromagnetic waves propagatingthrough a waveguide or other transmission media (e.g., light pulsespassing through a fiber-optic cable), or electrical signals transmittedthrough a wire.

Computer readable program instructions described herein can bedownloaded to respective computing/processing devices from a computerreadable storage medium or to an external computer or external storagedevice via a network, for example, the Internet, a local area network, awide area network and/or a wireless network. The network may comprisecopper transmission cables, optical transmission fibers, wirelesstransmission, routers, firewalls, switches, gateway computers and/oredge servers. A network adapter card or network interface in eachcomputing/processing device receives computer readable programinstructions from the network and forwards the computer readable programinstructions for storage in a computer readable storage medium withinthe respective computing/processing device.

Computer readable program instructions for carrying out operations ofthe present invention may be assembler instructions,instruction-set-architecture (ISA) instructions, machine instructions,machine dependent instructions, microcode, firmware instructions,state-setting data, configuration data for integrated circuitry, oreither source code or object code written in any combination of one ormore programming languages, including an object oriented programminglanguage such as Smalltalk, C++, or the like, and procedural programminglanguages, such as the “C” programming language or similar programminglanguages. The computer readable program instructions may executeentirely on the user's computer, partly on the user's computer, as astand-alone software package, partly on the user's computer and partlyon a remote computer or entirely on the remote computer or server. Inthe latter scenario, the remote computer may be connected to the user'scomputer through any type of network, including a local area network(LAN) or a wide area network (WAN), or the connection may be made to anexternal computer (for example, through the Internet using an InternetService Provider). In some embodiments, electronic circuitry including,for example, programmable logic circuitry, field-programmable gatearrays (FPGA), or programmable logic arrays (PLA) may execute thecomputer readable program instructions by utilizing state information ofthe computer readable program instructions to personalize the electroniccircuitry, in order to perform aspects of the present invention.

Aspects of the present invention are described herein with reference toflowchart illustrations and/or block diagrams of methods, apparatus(systems), and computer program products according to embodiments of theinvention. It will be understood that each block of the flowchartillustrations and/or block diagrams, and combinations of blocks in theflowchart illustrations and/or block diagrams, can be implemented bycomputer readable program instructions.

These computer readable program instructions may be provided to aprocessor of a general purpose computer, special purpose computer, orother programmable data processing apparatus to produce a machine, suchthat the instructions, which execute via the processor of the computeror other programmable data processing apparatus, create means forimplementing the functions/acts specified in the flowchart and/or blockdiagram block or blocks. These computer readable program instructionsmay also be stored in a computer readable storage medium that can directa computer, a programmable data processing apparatus, and/or otherdevices to function in a particular manner, such that the computerreadable storage medium having instructions stored therein comprises anarticle of manufacture including instructions which implement aspects ofthe function/act specified in the flowchart and/or block diagram blockor blocks.

The computer readable program instructions may also be loaded onto acomputer, other programmable data processing apparatus, or other deviceto cause a series of operational steps to be performed on the computer,other programmable apparatus or other device to produce a computerimplemented process, such that the instructions which execute on thecomputer, other programmable apparatus, or other device implement thefunctions/acts specified in the flowchart and/or block diagram block orblocks.

The flowchart and block diagrams in the Figures illustrate thearchitecture, functionality, and operation of possible implementationsof systems, methods, and computer program products according to variousembodiments of the present invention. In this regard, each block in theflowchart or block diagrams may represent a module, segment, or portionof instructions, which comprises one or more executable instructions forimplementing the specified logical function(s). In some alternativeimplementations, the functions noted in the blocks may occur out of theorder noted in the Figures. For example, two blocks shown in successionmay, in fact, be executed substantially concurrently, or the blocks maysometimes be executed in the reverse order, depending upon thefunctionality involved. It will also be noted that each block of theblock diagrams and/or flowchart illustration, and combinations of blocksin the block diagrams and/or flowchart illustration, can be implementedby special purpose hardware-based systems that perform the specifiedfunctions or acts or carry out combinations of special purpose hardwareand computer instructions.

Photography and videography-based applications have gained popularity inthe past decade for administrative and security purposes resulting inthe prevalence of security cameras and equipment in public areas.Existing applications may mostly be based on fixed imagery angles frompre-aligned cameras or cameras with limited movement. Furthermore,camera output may require manual observation to derive information or tomake decisions. Therefore, it may be advantageous to, among otherthings, provide a way to monitor crowds of people using a mobile imagecapturing device capable of capturing images from an altitude, such asan aircraft with a camera, and automatically analyze the camera outputto make decisions in response to the characteristics of the crowd.

The following described exemplary embodiments provide a system, methodand program product for monitoring a crowd using a device for capturingimages from an altitude, such as aircraft and aerial unmanned vehicles(UAVs) or aerial drones. As such, the present embodiment has thecapacity to improve the technical field of crowd analysis by monitoringcrowds from images taken by aerial drones that may dynamically traverseabove an area and make decisions in real-time. More specifically, astill image camera and a video camera mounted to the bottom of acontrolled flight-capable aerial drone or other aircraft that may alsohave the capability to wirelessly transmit data of the generated imagesand video of a group of persons forming a crowd. From the static imagestaken by the drone, an image partitioning algorithm may be used toidentify individuals within the crowd images. Then a pre-trained modelmay be used to predict the gender, age, socioeconomic status, or otherattributes of the individuals from the partitioned images. From thecaptured video, an image partitioning algorithm may be used to identifyindividuals and thereby determine the rate of entry and rate of exitfrom a certain defined geographic area. Additionally, motion points forindividuals may be plotted to determine the distribution of the mass ofpeople gathering from the overhead video. Based on predicted crowdcharacteristics drawn from predicted individual characteristics,decisions may be made in real-time for advertising, security, or someother purpose.

The present embodiment may be used to derive insights from a massgathering of people by categorizing the mass of people and analyzing themotion of the mass by zoning a geographic area. Drones equipped withflight and imagery capabilities may be used to generate the data foranalyzing the mass of people. By using aircraft, overhead images may begenerated that provide a better position to assess the characteristicsof the crowd and track individuals within the crowd. Furthermore,aircraft may have fewer obstacles to navigate around to capture imagesof the crowd. The present embodiment may be useful for securityenforcement by reducing the number of personnel that may need to bedeployed to monitor crowds. The present embodiment may also helpadvertisers to analyze a target mass of individuals and performcategorized advertisement that may be more effective given the derivedcrowd characteristics.

According to at least one embodiment, the operational area over a massgathering is first determined. Then, the target geographical area may bedefined by a polygon with coordinates. Next, the drone takes the area asinput and partitions the area into smaller zones. The drone thentraverses the area using a traversing algorithm to move and position thedrone over the zones according to the algorithm. For each zone, thedrone captures images (e.g., 2-5 images) at a predefined time interval(e.g., 2-10 seconds). Thereafter, image processing steps may beperformed for each of the images in the sequence of captured images.Images may be partitioned to create top-level images of each individualwithin the crowd. A trained model may then process the partitionedimages to predict features (e.g., gender, age, and direction the personis facing) of every individual. Best match algorithms may then be usedto identify every individual across the sequence of captured images.From the sequential images or from captured video, the motion directionof each individual may be determined. Then, aggregate motioninformation, categorization, and concentration of the entire crowd maybe determined from captured still images or video based on analysis ofthe constituent individuals within the crowd and observing the movementof the crowd as a whole. Next, according to one embodiment, businessintelligence may be used to decide placement and duration of dynamicadvertisements. According to at least one other embodiment, expertknowledge may be used to decide administrative control and measures overthe mass gathering of people.

Referring to FIG. 1, an exemplary networked computer environment 100 inaccordance with one embodiment is depicted. The networked computerenvironment 100 may include a computer 102 with a processor 104 and adata storage device 106 that is enabled to run a software program 108and a crowd analysis program 110 a. The networked computer environment100 may also include a server 112 that is enabled to run a crowdanalysis program 110 b that may interact with a database 114 and acommunication network 116. The networked computer environment 100 mayinclude a plurality of computers 102 and servers 112, only one of whichis shown. The communication network 116 may include various types ofcommunication networks, such as a wide area network (WAN), local areanetwork (LAN), a telecommunication network, a wireless network, a publicswitched network and/or a satellite network. It should be appreciatedthat FIG. 1 provides only an illustration of one implementation and doesnot imply any limitations with regard to the environments in whichdifferent embodiments may be implemented. Many modifications to thedepicted environments may be made based on design and implementationrequirements.

The client computer 102 may communicate with the server computer 112 viathe communications network 116. The communications network 116 mayinclude connections, such as wire, wireless communication links, orfiber optic cables. As will be discussed with reference to FIG. 4,server computer 112 may include internal components 902 a and externalcomponents 904 a, respectively, and client computer 102 may includeinternal components 902 b and external components 904 b, respectively.Server computer 112 may also operate in a cloud computing service model,such as Software as a Service (SaaS), Platform as a Service (PaaS), orInfrastructure as a Service (IaaS). Server 112 may also be located in acloud computing deployment model, such as a private cloud, communitycloud, public cloud, or hybrid cloud. Client computer 102 may be, forexample, a mobile device, a telephone, a personal digital assistant, anetbook, a laptop computer, a tablet computer, a desktop computer, anaerial drone flight computer, or any type of computing devices capableof running a program, accessing a network, and accessing a database 114.According to various implementations of the present embodiment, thecrowd analysis program 110 a, 110 b may interact with a database 114that may be embedded in various storage devices, such as, but notlimited to a computer/mobile device 102, a networked server 112, or acloud storage service.

According to the present embodiment, a user using a client computer 102or a server computer 112 may use the crowd analysis program 110 a, 110 b(respectively) to monitor crowds of people using images taken from anaerial drone and analyze the images of the crowd to determine risks ortargeted advertisements in real-time. The crowd analysis method isexplained in more detail below with respect to FIG. 3.

Referring now to FIG. 2, a networked aerial image capture device system200 according to at least one embodiment is depicted. The networkedaerial image capture device system 200 may include a server 112 runningthe crowd analysis program 110 b, a communication network 116, and anaerial drone 202 (i.e., aerial image capturing device). The aerial drone202 may include a flight computer 204, a wireless adapter 206, a stillcamera 208, and a video camera 210.

The crowd analysis program 110 b running on the server 112 maycommunicate via the communication network 116 with the wireless adapter206 of the aerial drone 202. The communication network 116 may includewireless connections, such as wi-fi or satellite communication. Forexample, the crowd analysis program 110 b may generate a flight path andtransmit the generated flight path using the communication network 116to the wireless adapter 206 within the aerial drone 202. The wirelessadapter 206 may send and receive data wirelessly using the communicationnetwork 116 and the wireless adapter 206 may also interact with theflight computer 204.

The flight computer 204 may be a computer 102 designed to control theaerial drone 202 in flight by keeping the aerial drone 202 level andflying the aerial drone 202 according to a flight path. Furthermore, theflight computer 204 may control onboard sensors, such as the stillcamera 208 and video camera 210 attached to the bottom of the aerialdrone 202. The flight computer 204 may send the still camera 208 and thevideo camera 210 instructions to move to point in a specific directionand when to capture images. Images captured by the still camera 208 andthe video camera 210 may be sent to the flight computer 204 for storageand transmission. The flight computer 204 may then relay the capturedimages to the wireless adapter 206. Then, the wireless adapter 206 maytransmit the images using the communication network 116 to the crowdanalysis program 110 b running on the server 112 for analysis.

Referring now to FIG. 3, an operational flowchart illustrating theexemplary crowd analysis process 300 used by the crowd analysis program110 a and 110 b according to at least one embodiment is depicted.

At 302, an operational geographic area over a crowd of gathered peopleis determined. The operational geographic area may be determined basedon input from a user, a warning from fixed sensors near the geographicarea, and so on. For example, a city park may be designated as theoperational geographic area by a user.

Next, at 304, the geographic area is defined by a polygon. Using knownmethods, the operational geographic area may be represented as a polygonwith a set of geographic coordinates. For example, if the city park thatwas designated as the geographic area is approximately rectangular inshape, then the geographic coordinates of the four corner points may bedetermined and saved as a set of geographic coordinates. From the savedgeographic coordinates, a polygon corresponding with the city park maybe defined that may be used as input into drone controlling software.

Then, at 306, the polygon representing the geographic area ispartitioned into zones. The geographic area may be partitioned intosmaller geographic subdivisions or zones to efficiently analyze thetotal operational geographic area. The partitioning may include purehexagonal or Cairo pentagonal zones and a set of coordinates may bedetermined for the aerial drone 202 to be positioned for each zone.Continuing the previous example, the city park may be subdivided intofour zones, such as zones Z₁, Z₂, Z₃, and Z₄, with each zone having atleast one position coordinate associated with the zone.

At 308, a flight path over the zones is determined. Once the geographicarea is subdivided into zones as described previously at 306, a flightpath may be determined for the aerial drone 202 to fly over all of thezones, thus covering the original geographic area. The aerial drone 202may start from a point close to the area of operation. A best routealgorithm may be used to generate the flight path for the aerial drone202 to visit each position coordinate within the set of positioncoordinates. For example, if the area of operation for the aerial drone202 is nearest Z₃, then, based on executing a best route algorithm, thedetermined flight path may create a path from Z₃ to Z₂, from Z₂ to Z₄,and then from Z₄ to Z₁. More specifically, the determined flight pathmay plot the flight path to the position coordinate of Z₃, then to theposition coordinate for Z₂, then to the position coordinate for Z₄, andfinally to the position coordinate for Z₁. Furthermore, the height oraltitude of the aerial drone 202 may be set as part of the flight pathwith differing altitudes at various points along the flight path tocircumvent obstacles or provide a more appropriate image resolutiongiven a zone size.

Next, at 310, the aerial drone 202 will fly according to the determinedflight path and capture images of the crowd of people. As describedpreviously, a server 112 running the crowd analysis program 110 b maywirelessly transmit the determined flight path to the wireless adapter206 of the aerial drone 202 using the communication network 116. Theaerial drone 202 may perform self-controlled operation (i.e., autonomousflying) according to the received flight path using the flight computer204. As the aerial drone 202 flies along the determined flight path,still images may be captured at predefined time intervals using thestill camera 208 and video may be captured simultaneously using thevideo camera 210. Additionally, the aerial drone 202 may loop throughthe flight path continuously throughout the aerial drone's 202 operatingtime. As the aerial drone 202 flies to a position coordinate, apredefined number of still images, such as two, may be captured at apredefined time interval, such as two seconds apart, in addition tocapturing video footage before the drone moves on to the next positioncoordinate in the flight path. The aerial drone's 202 results may thenbe displayed in a dashboard or other drone software.

Then, at 312, the captured images are partitioned. The captured imagesmay be transmitted wirelessly to a server 112 for processing from thewireless adapter 206 of the aerial drone 202 using a communicationnetwork 116. At the server 112, the images may be partitioned using apartitioning algorithm to create images corresponding with each uniqueindividual within the set of images generated by the aerial drone 202.The partitioning algorithm may identify geometry or other patternsconsistent with the silhouette of a person or by detecting featuresconsistent with a face, or by some other method. The image may then bepartitioned by cropping or delineating a region within an image toinclude a single individual per partition. For example, a still imagefrom zone Z₂ may be processed using a partitioning algorithm thatidentifies twenty individuals. The partitioning algorithm would thencreate twenty partitions from the still image with each partitionincluding a single individual. Image partitioning may be performed in alike manner for each image of every zone until all images have beensimilarly partitioned.

At 314, the characteristics of the individuals in the partitioned imagesare predicted. Using a pre-trained machine learning model, individualpartitioned images may be analyzed to determine characteristics of theperson within each partitioned image. Determined characteristics mayinclude, for example, demographics, such as age and gender.Additionally, the determined characteristics may include other physicalcharacteristics, such as height, weight, and direction the individual isfacing, and other characteristics about an individual that may bederived from analyzing an image. Furthermore, based on the location andorientation of the aerial drone 202 when the picture was taken, thegeographic location of the area being photographed, and the position ofthe individual within the image, the geographic location of theindividual may be determined. The determined characteristics may then besaved as metadata. Thereafter, the images may be appended with themetadata, including the individual's age, gender, facing direction,location, and so on. Alternatively, the metadata may be saved in a datastructure, such as an array, with a pointer or other indicator to anindividual in a partitioned image.

Next, at 316, the motion of the individuals in the images is determined.By comparing the position of individuals from one partitioned image to asecond partitioned image of the same individual, the movement directionand speed of each individual may be tracked to determine an individualmovement vector indicating the movement direction and speed of theindividual. For example, if an individual appears in the center of afirst image and the same individual appears in the bottom-right portionin a second image, then the individual has moved in a northeastdirection based on the known position and orientation of the aerialdrone 202 and still camera 208 when the images were generated.Alternatively, by comparing the time the still images were taken to thesame time in the video footage, individuals may be identified within thevideo since the still camera 208 and the video camera 210 may be mountedin close proximity and therefore individuals may appear in similarpositions in the still images and the video generated at the same time.Thus, using the video, an individual's movement may be determined bytracking the change in the position of the individual in the videofootage.

Then, at 318, the aggregate motion, categorization, and concentration ofthe crowd may be determined. Based on the movement of the individualswithin the crowd, the composite motion of the crowd in general may alsobe determined. For example, by comparing the movement of multipleindividuals, a point of convergence may be determined at a centrallocation or a direction the crowd of people may be moving towards thatmay be expressed as a crowd movement vector. Furthermore, the crowdconcentration may be determined based on each individual's positionwithin the defined geographic area which may then be plotted as points.Based on the density of the plotted points and movement of pointscorresponding to individuals, the mass movement of the crowd may also bededuced by the rate of entry and exit from a certain area of the zone.Furthermore, based on the characteristics of the individuals determinedat 314, the characteristics of the collective crowd may be determined tocategorize the crowd. For example, if 88% of the individuals areestimated to be within the ages of 16 to 25, the crowd may becategorized as a crowd of a younger demographic of people.

At 320, an action is taken in response to the information determinedabout the crowd. Business intelligence may be applied to the aggregatemotion, categorization, and concentration of the crowd. Based on theresults from the applied business intelligence, an advertiser may decideon a particular advertising strategy and advertise to the crowd inreal-time. Continuing the previous example, if the crowd was categorizedas a crowd of younger people, specific advertising may be delivered tothe individuals in the crowd that is tailored for individuals within thecrowd's age demographic. The advertising may be electronically deliveredto the mobile devices of the individuals within the crowd, or bycontacting personnel amongst the crowd by radio, text message, and soforth to distribute tangible advertising, such as flyers, that may bemost effective given the crowd characteristics.

Alternatively, the characteristics of the crowd, density, and movementmay raise security concerns and police or military may respond tocontrol the crowd, or move sensitive items or personnel for protectionor to reduce any hostilities. In security applications, securityintelligence may be applied to the aggregate motion, categorization, andconcentration of the crowd. The resulting security intelligence analysisof the crowd information may be used by security personnel to decidewhich security measures should be taken and the extent, duration, andpositioning of the security measures. In another scenario, eventorganizers may use the resulting crowd information to plan how toefficiently handle the people within the crowd or how to make changes tominimize future crowd formation and congestion.

It may be appreciated that FIGS. 2 and 3 provide only an illustration ofone embodiment and do not imply any limitations with regard to howdifferent embodiments may be implemented. Many modifications to thedepicted embodiment(s) may be made based on design and implementationrequirements.

FIG. 4 is a block diagram 900 of internal and external components ofcomputers depicted in FIG. 1 in accordance with an illustrativeembodiment of the present invention. It should be appreciated that FIG.4 provides only an illustration of one implementation and does not implyany limitations with regard to the environments in which differentembodiments may be implemented. Many modifications to the depictedenvironments may be made based on design and implementationrequirements.

Data processing system 902, 904 is representative of any electronicdevice capable of executing machine-readable program instructions. Dataprocessing system 902, 904 may be representative of a smart phone, acomputer system, PDA, or other electronic devices. Examples of computingsystems, environments, and/or configurations that may represented bydata processing system 902, 904 include, but are not limited to,personal computer systems, server computer systems, thin clients, thickclients, hand-held or laptop devices, multiprocessor systems,microprocessor-based systems, network PCs, minicomputer systems, anddistributed cloud computing environments that include any of the abovesystems or devices.

User client computer 102 and network server 112 may include respectivesets of internal components 902 a, b and external components 904 a, billustrated in FIG. 4. Each of the sets of internal components 902 a, bincludes one or more processors 906, one or more computer-readable RAMs908, and one or more computer-readable ROMs 910 on one or more buses912, and one or more operating systems 914 and one or morecomputer-readable tangible storage devices 916. The one or moreoperating systems 914, the software program 108 and the crowd analysisprogram 110 a in client computer 102, and the crowd analysis program 110b in network server 112, may be stored on one or more computer-readabletangible storage devices 916 for execution by one or more processors 906via one or more RAMs 908 (which typically include cache memory). In theembodiment illustrated in FIG. 4, each of the computer-readable tangiblestorage devices 916 is a magnetic disk storage device of an internalhard drive. Alternatively, each of the computer-readable tangiblestorage devices 916 is a semiconductor storage device such as ROM 910,EPROM, flash memory or any other computer-readable tangible storagedevice that can store a computer program and digital information.

Each set of internal components 902 a, b also includes a R/W drive orinterface 918 to read from and write to one or more portablecomputer-readable tangible storage devices 920 such as a CD-ROM, DVD,memory stick, magnetic tape, magnetic disk, optical disk orsemiconductor storage device. A software program, such as the softwareprogram 108 and the crowd analysis program 110 a and 110 b can be storedon one or more of the respective portable computer-readable tangiblestorage devices 920, read via the respective R/W drive or interface 918,and loaded into the respective hard drive 916.

Each set of internal components 902 a, b may also include networkadapters (or switch port cards) or interfaces 922 such as a TCP/IPadapter cards, wireless wi-fi interface cards, or 3G or 4G wirelessinterface cards or other wired or wireless communication links. Thesoftware program 108 and the crowd analysis program 110 a in clientcomputer 102 and the crowd analysis program 110 b in network servercomputer 112 can be downloaded from an external computer (e.g., server)via a network (for example, the Internet, a local area network or other,wide area network) and respective network adapters or interfaces 922.From the network adapters (or switch port adaptors) or interfaces 922,the software program 108 and the crowd analysis program 110 a in clientcomputer 102 and the crowd analysis program 110 b in network servercomputer 112 are loaded into the respective hard drive 916. The networkmay comprise copper wires, optical fibers, wireless transmission,routers, firewalls, switches, gateway computers and/or edge servers.

Each of the sets of external components 904 a, b can include a computerdisplay monitor 924, a keyboard 926, and a computer mouse 928. Externalcomponents 904 a, b can also include touch screens, virtual keyboards,touch pads, pointing devices, and other human interface devices. Each ofthe sets of internal components 902 a, b also includes device drivers930 to interface to computer display monitor 924, keyboard 926, andcomputer mouse 928. The device drivers 930, R/W drive or interface 918,and network adapter or interface 922 comprise hardware and software(stored in storage device 916 and/or ROM 910).

It is understood in advance that although this disclosure includes adetailed description on cloud computing, implementation of the teachingsrecited herein are not limited to a cloud computing environment. Rather,embodiments of the present invention are capable of being implemented inconjunction with any other type of computing environment now known orlater developed.

Cloud computing is a model of service delivery for enabling convenient,on-demand network access to a shared pool of configurable computingresources (e.g., networks, network bandwidth, servers, processing,memory, storage, applications, virtual machines, and services) that canbe rapidly provisioned and released with minimal management effort orinteraction with a provider of the service. This cloud model may includeat least five characteristics, at least three service models, and atleast four deployment models.

Characteristics are as follows:

On-demand self-service: a cloud consumer can unilaterally provisioncomputing capabilities, such as server time and network storage, asneeded automatically without requiring human interaction with theservice's provider.

Broad network access: capabilities are available over a network andaccessed through standard mechanisms that promote use by heterogeneousthin or thick client platforms (e.g., mobile phones, laptops, and PDAs).

Resource pooling: the provider's computing resources are pooled to servemultiple consumers using a multi-tenant model, with different physicaland virtual resources dynamically assigned and reassigned according todemand. There is a sense of location independence in that the consumergenerally has no control or knowledge over the exact location of theprovided resources but may be able to specify location at a higher levelof abstraction (e.g., country, state, or datacenter).

Rapid elasticity: capabilities can be rapidly and elasticallyprovisioned, in some cases automatically, to quickly scale out andrapidly released to quickly scale in. To the consumer, the capabilitiesavailable for provisioning often appear to be unlimited and can bepurchased in any quantity at any time.

Measured service: cloud systems automatically control and optimizeresource use by leveraging a metering capability at some level ofabstraction appropriate to the type of service (e.g., storage,processing, bandwidth, and active user accounts). Resource usage can bemonitored, controlled, and reported providing transparency for both theprovider and consumer of the utilized service.

Service Models are as follows:

Software as a Service (SaaS): the capability provided to the consumer isto use the provider's applications running on a cloud infrastructure.The applications are accessible from various client devices through athin client interface such as a web browser (e.g., web-based e-mail).The consumer does not manage or control the underlying cloudinfrastructure including network, servers, operating systems, storage,or even individual application capabilities, with the possible exceptionof limited user-specific application configuration settings.

Platform as a Service (PaaS): the capability provided to the consumer isto deploy onto the cloud infrastructure consumer-created or acquiredapplications created using programming languages and tools supported bythe provider. The consumer does not manage or control the underlyingcloud infrastructure including networks, servers, operating systems, orstorage, but has control over the deployed applications and possiblyapplication hosting environment configurations.

Infrastructure as a Service (IaaS): the capability provided to theconsumer is to provision processing, storage, networks, and otherfundamental computing resources where the consumer is able to deploy andrun arbitrary software, which can include operating systems andapplications. The consumer does not manage or control the underlyingcloud infrastructure but has control over operating systems, storage,deployed applications, and possibly limited control of select networkingcomponents (e.g., host firewalls).

Deployment Models are as follows:

Private cloud: the cloud infrastructure is operated solely for anorganization. It may be managed by the organization or a third party andmay exist on-premises or off-premises.

Community cloud: the cloud infrastructure is shared by severalorganizations and supports a specific community that has shared concerns(e.g., mission, security requirements, policy, and complianceconsiderations). It may be managed by the organizations or a third partyand may exist on-premises or off-premises.

Public cloud: the cloud infrastructure is made available to the generalpublic or a large industry group and is owned by an organization sellingcloud services.

Hybrid cloud: the cloud infrastructure is a composition of two or moreclouds (private, community, or public) that remain unique entities butare bound together by standardized or proprietary technology thatenables data and application portability (e.g., cloud bursting forload-balancing between clouds).

A cloud computing environment is service oriented with a focus onstatelessness, low coupling, modularity, and semantic interoperability.At the heart of cloud computing is an infrastructure comprising anetwork of interconnected nodes.

Referring now to FIG. 5, illustrative cloud computing environment 1000is depicted. As shown, cloud computing environment 1000 comprises one ormore cloud computing nodes 100 with which local computing devices usedby cloud consumers, such as, for example, personal digital assistant(PDA) or cellular telephone 1000A, desktop computer 1000B, laptopcomputer 1000C, and/or automobile computer system 1000N may communicate.Nodes 100 may communicate with one another. They may be grouped (notshown) physically or virtually, in one or more networks, such asPrivate, Community, Public, or Hybrid clouds as described hereinabove,or a combination thereof. This allows cloud computing environment 1000to offer infrastructure, platforms and/or software as services for whicha cloud consumer does not need to maintain resources on a localcomputing device. It is understood that the types of computing devices1000A-N shown in FIG. 5 are intended to be illustrative only and thatcomputing nodes 100 and cloud computing environment 1000 can communicatewith any type of computerized device over any type of network and/ornetwork addressable connection (e.g., using a web browser).

Referring now to FIG. 6, a set of functional abstraction layers 1100provided by cloud computing environment 1000 is shown. It should beunderstood in advance that the components, layers, and functions shownin FIG. 6 are intended to be illustrative only and embodiments of theinvention are not limited thereto. As depicted, the following layers andcorresponding functions are provided:

Hardware and software layer 1102 includes hardware and softwarecomponents. Examples of hardware components include: mainframes 1104;RISC (Reduced Instruction Set Computer) architecture based servers 1106;servers 1108; blade servers 1110; storage devices 1112; and networks andnetworking components 1114. In some embodiments, software componentsinclude network application server software 1116 and database software1118.

Virtualization layer 1120 provides an abstraction layer from which thefollowing examples of virtual entities may be provided: virtual servers1122; virtual storage 1124; virtual networks 1126, including virtualprivate networks; virtual applications and operating systems 1128; andvirtual clients 1130.

In one example, management layer 1132 may provide the functionsdescribed below. Resource provisioning 1134 provides dynamic procurementof computing resources and other resources that are utilized to performtasks within the cloud computing environment. Metering and Pricing 1136provide cost tracking as resources are utilized within the cloudcomputing environment, and billing or invoicing for consumption of theseresources. In one example, these resources may comprise applicationsoftware licenses. Security provides identity verification for cloudconsumers and tasks, as well as protection for data and other resources.User portal 1138 provides access to the cloud computing environment forconsumers and system administrators. Service level management 1140provides cloud computing resource allocation and management such thatrequired service levels are met. Service Level Agreement (SLA) planningand fulfillment 1142 provide pre-arrangement for, and procurement of,cloud computing resources for which a future requirement is anticipatedin accordance with an SLA.

Workloads layer 1144 provides examples of functionality for which thecloud computing environment may be utilized. Examples of workloads andfunctions which may be provided from this layer include: mapping andnavigation 1146; software development and lifecycle management 1148;virtual classroom education delivery 1150; data analytics processing1152; transaction processing 1154; and crowd analysis 1156. A crowdanalysis program 110 a, 110 b provides a way to monitor crowds of peopleusing images taken at an altitude above the crowd from an imagecapturing device and analyze the images of the crowd to determine risksor targeted advertisements in real-time.

The descriptions of the various embodiments of the present inventionhave been presented for purposes of illustration, but are not intendedto be exhaustive or limited to the embodiments disclosed. Manymodifications and variations will be apparent to those of ordinary skillin the art without departing from the scope of the describedembodiments. The terminology used herein was chosen to best explain theprinciples of the embodiments, the practical application or technicalimprovement over technologies found in the marketplace, or to enableothers of ordinary skill in the art to understand the embodimentsdisclosed herein.

1. A method for analyzing a crowd using a plurality of images capturedby an aerial drone, the method comprising: determining a geographic areaassociated with the crowd; partitioning the determined geographic areainto a plurality of zones; determining a flight path covering each zonewithin the plurality of zones; sending the determined flight path to theaerial drone; flying, by the aerial drone, along the sent flight path;generating, by the aerial drone, the plurality of images, and aplurality of drone position data that comprises a drone location, adrone orientation, and a photographed location; receiving the pluralityof images and the plurality of drone position data from the aerialdrone; analyzing the received plurality of images to identify aplurality of individuals associated with the crowd, wherein analyzingthe received plurality of images to identify the plurality ofindividuals associated with the crowd further comprises cropping eachimage within the received plurality of images to create an imagepartition for each identified individual within the identified pluralityof individuals; determining an individual position for each identifiedindividual within the identified plurality of individuals based on theanalyzed plurality of images and the received plurality of droneposition data; determining an individual movement vector for eachidentified individual within the identified plurality of individualsbased on tracking changes in the determined individual positionassociated with the individual as captured within the analyzed pluralityof images; predicting a plurality of individual characteristics for eachidentified individual based on the analyzed plurality of images, whereinpredicting the plurality of individual characteristics comprises using apre-trained machine learning model to process the analyzed plurality ofimages and predict the plurality of individual characteristics, andwherein the predicted plurality of individual characteristics includes aplurality of demographic characteristics and a plurality of physicalcharacteristics; predicting a plurality of crowd characteristics basedon the predicted plurality of individual characteristics for theplurality of identified individuals, wherein the predicted plurality ofcrowd characteristics includes an aggregate motion characteristicdetermined based on the determined individual movement vector for eachidentified individual and a rate of entry and a rate of exitcorresponding with each zone within the plurality of zones, and whereinthe predicted plurality of crowd characteristics includes a crowdconcentration characteristic based on the determined individual positionfor each identified individual; and determining a placement and aduration for dynamic advertising to distribute to the crowd based on thepredicted plurality of crowd characteristics.